Method and apparatus for using metadata from different sources

ABSTRACT

Metadata from different sources are used to organize recordings, search for content and access specific recordings. Due to different metadata sources using different standards, the metadata have different semantics and syntax. According to the invention, from the original metadata received from the different sources unified metadata are derived using one or more transformation rules. The transformation rules specify how the content or parts of the content of said original metadata shall be used to constitute the content or part of the content of said unified metadata. The derived unified metadata are presented to a user and processed in response to a user input.

The invention relates to a method and to an apparatus for using metadatafrom different sources, especially for organizing of recordings,searching for content and accessing specific recordings.

BACKGROUND

The capacity of digital storage media for personal video recording growscontinuously. Currently it is possible to store about 20 full-lengthmovies on a single 100 gigabyte (GB) hard disk. In 2005, it will likelybe possible to store about 80 movies on a single 400 GB hard disk.

Similarly, the capacity of exchangable storage media is expanding. DVDscontaining full-length movies are widely available and also rewritableDVDs (DVD-RAM, DVD-RW, DVD+RW) are on the market, having a storagecapacity of 4.7 GB. Furthermore, the basic specification for a nextgeneration of large capacity optical discs called “Blu-ray Disc” hasrecently been established, enabling the recording, rewriting andplayback of up to 27 GB of data on a single sided single layer disc. Inorder to further increase the capacity, two or more layers may be usedper side and these may be applied to both sides of the disc. Finally,several discs may be combined in a special magazine.

This enormous amount of data requires new ways to organize therecordings, search for content and access specific recordings. Onepossible solution for this is to use so-called metadata, defined as dataabout data, for the recorded content. Various industry groups andstandard bodies have been developing metadata standards for differentpurposes and applications. In the context of metadata usage, the contentto which metadata are related is often called “essence”.

The compression standards for AV streams MPEG-1, MPEG-2 and MPEG-4already contain some program specific information (PSI), specified inthe respective MPEG systems standard. Similarly, the DVB standard usedfor the transmission of digital television signals specifies ServiceInformation (DVB-SI) included in a DVB compliant MPEG-2 transport streammultiplex.

More recently, the TV-Anytime Forum defined in the specification S-3 onMetadata, document number SP003v1.1 metadata which allow the consumer tofind, navigate and manage content from a variety of internal andexternal sources including, for example, enhanced broadcast, interactiveTV, Internet and local storage. Additionally, these metadata may alsoinclude information about user preferences, such as favorite actors orTV shows, which facilitate automatic filtering and acquisition ofcontent by agents on behalf of the consumer.

Similarly, an MPEG standard for the effective and efficient access tomultimedia content is currently under development [ISO/IEC 15938:Information Technology—Multimedia content description interface]. ThisMPEG-7 standard will offer metadata elements and their structure andrelationships, that are defined in the form of Descriptors andDescription Schemes to create descriptions of the multimedia content.

INVENTION

The invention is based on the recognition of the following fact. Oftenseveral different metadata describing the same content may be available.However, these metadata may have different associated meanings(semantics) and ways of coding (syntax), which are defined in therespective standards, e.g. the MPEG-7 or TV-Anytime standard. On theother hand, in systems planning to employ metadata, certain semanticsand syntax of metadata or metadata fields will be more appropriate thanothers, depending on the properties and features the system shall beable to perform. Therefore, it is desirable to allow the use of allavailable metadata, despite their different semantics and syntax.

Therefore, a problem to be solved by the invention is to allow the useof metadata originating from different sources having differentsemantics or syntax.

This problem is solved by the method disclosed in claim 1. An apparatusthat utilizes this method is disclosed in claim 8.

According to the invention original metadata are received from differentsources and unified metadata are derived from said original metadatausing one or more transformation rules, wherein the transformation rulesspecify how the content or parts of the content of said originalmetadata shall be used to constitute the content or part of the contentof said unified metadata.

Advantageously, the derived unified metadata are presented to a user andare processed in response to a user input.

Advantageously, the processing results in selecting, deleting ormodifying said unified metadata.

Furthermore, it is of advantage to store the original metadata, keep thestored original metadata unchanged and enable to recurr from the unifiedmetadata to the original metadata.

For the transformation rules, it is advantageous to base them on formaland/or structural properties of the original metadata in all cases wherethe original metadata are themself structured.

For unstructured original metadata of textual nature, it is advantageousto base the transformation rules on a similarity measure which allowsfor a tolerant matching of phrases against a user-defined search phrase.

According to a further advantageous embodiment a part of thetransformation rules covers the mapping or inheritance of the essencelinks from within the original metadata into all derived unifiedmetadata.

Further advantageous embodiments of the invention result from thefollowing description.

DRAWING

Exemplary embodiments of the invention are described with reference tothe accompanying drawings, which show in:

FIG. 1 generically the conversion of original metadata records withdifferent syntax and semantics into unified metadata records;

FIG. 2. a block diagram of the processing of metadata including thederivation of unified metadata from received original metadata.

EXEMPLARY EMBODIMENTS

FIG. 1 shows generically the conversion of original metadata recordswith different syntax and semantics into unified metadata records, inthe following also called Core Metadata. Transformations T1, . . . , Tj,. . . specify rules how the content or parts of the content of theoriginal metadata records of a predetermined class shall be used toconstitute the content or part of the content of a unified metadatarecord. In FIG. 1, the blocks “ODR1” to “ODRk+m” symbolize differentclasses of original metadata records, and more specifically, it shall beassumed that blocks ODR1 and ODRk both are governed by one specificsyntax/semantics, whereas blocks ODRk+1 and ODRk+m are governed by asecond, different syntax/semantics. All original metadata records ofclass ODR1 are converted by a set of transformations T1 into instancesof a first class of unified metadata records UDR1. As part of thisconversion, the transformation T1 may mandate the presence of certainparts within the instance of ODR1, so that by applying T1, not allinstances of ODR1 will actually generate an output UDR1. Also, it mustbe kept in mind, that the conversion described by T1 may mandate thatonly certain parts of the instance of ODR1 shall be taken to constitutecertain parts of the emerging instance of UDR1. Similarly, the same setof transformations T1 may contain a rule which defines that, undercertain conditions, some predefined parts of instances of a second classof original metadata records ODRk are converted into other instances ofthe first kind of unified metadata records UDR1. However, otherinstances of the second class of original metadata ODRk, if they containcertain other parts, are converted by a different set of transformationsTj into instances of a second class of unified metadata records UDRj.Similarly, further original metadata records of classes ODRk+1 andODRk+m having a syntax/semantics according to a second standard areconverted by the transformations T1 and Tj into instances of the firstand second unified metadata records UDR1 and UDRj. In this way, when asufficient number of original metadata is available and the respectivetransformation rules are applied, several candidates will be generatedfor each kind of Core Metadata. In this way, a user has sufficientmaterial from which he can editorially distill the Core Metadata hefinally wants.

FIG. 2 shows an example for the derivation of Core Metadata fromoriginally received metadata as part of an overall metadata processingchain. Although the further description concentrates on the processingof an MPEG-2 transport stream, it can easily be generalised for use inany system using multiplexed bitstreams comprising metadata.

When recording a broadcast DVB transport stream TS additional serviceinformation encompassed in the multiplex besides the audio and videosignals is parsed from the multiplex. For this purpose the broadcastsignal is analyzed for metadata like teletext, DVB-SI etc. by a DVBmetadata parser MD-PAR and the detected metadata are demultiplexed andextracted.

The extracted metadata are transformed into XML metadata descriptorsXML-DS. This XML representation for the metadata descriptors allows tohave a generic framework for different kinds of metadata descriptors. Inother words, the metadata database MD-DB can store any kind of metadataas long as there is a transformer that converts it from its nativerepresentation into the XML representation. Similarly it is possible toperform queries on the metadata, independent of their respectivemetadata sources.

There are different transformers for different metadata sources, likee.g. a transformer XML-SI for the transformation of DVB-SI data or afurther transformer XML-TT for the transformation of Teletext or ClosedCaption data. The transformers transform the metadata from their nativecoding, e.g. DVB-SI binary format, into an XML representation. Thistransformation is keeping the information inside the metadata “as is”.It neither adds nor removes information. This allows a very flexiblelater use of the metadata. During recording the metadata descriptors areaccumulated and stored into the metadata database MD-DB.

Similarly, metadata descriptors generated manually by the user such asuser-annotations UA may be stored after they have been transformed by atransformer XML-UA into XML representation. Furthermore, metadatadescriptors according to other standards like HTML, TV-Anytime, MPEG-7or SMPTE are transformed by respective transformers XML-HTML, XML-TVA,XML-M7 or XML-SMPTE and stored. These metadata descriptors may bereceived via the same or other transmission paths like the Internet.

An archive system may also store metadata descriptors originating fromother recording devices and/or storage media. However, this necessitatesan export and import facility MD-EXP-IMP for metadata descriptors, suchthat metadata descriptors can be exchanged between devices and/or media.

In order to use the metadata descriptors stored in the metadata databaseMD-DB the inventive conversion of the original metadata into CoreMetadata CM has to be performed using specific transformation rulesMD-T. Here is where it becomes beneficial that the metadata is stored“as is”. No information has been added or removed from the complete setof metadata information that is available.

Finally, access may be provided to all metadata via a user interface UIwith query and browsing techniques Q for finding the correspondingvideo/audio stream or metadata the selected metadata descriptor ispointing to.

One example of the conversion of the original metadata into CoreMetadata is the conversion of different metadata items into titleinformation for a given recording. Often the title information is themost useful and versatile single metadata item when it comes toarchiving, searching and sorting of any kind of AV material. For allmaterial, be it self recorded (e.g. with a camcorder), or recorded frombroadcast, or be it on canned media, mostly everybody will be able toassociate a Title with it, under which the material can, on one hand, berecognized for later search, but, on the other hand, can also besufficiently differentiated, if necessary, against other material whichmight be closely similar.

Corresponding to this very intuitive notion of Title, many existingmetadata systems comprise items which constitute a container for Titles,although they may come under different names and have distinct syntaxes.

-   -   DVB, in its “DVB-SI” system of Service Information, defines a        “short event descriptor” which “provides the name of the event        and a short description of the event in text form”. The syntax        of the short event descriptor foresees two variable length text        fields: a dedicated one for the “event_name”, and another one to        “specify the text description of the event”.    -   MPEG-7, in its “Creation Description Scheme”, has the notion of        “Title”, as “the name of an audio-visual program”.    -   TVAnytime, in its work on Electronic Program Guides, mentions        the concepts of “Title” as “Textual title of the program ( . . .        ) Multiple title descriptors may be included”. Similar concepts        of “Episode Title” and “Series Title” exist.    -   Any container for any kind of User Annotations might always        happen to be used for a user-specific Title of a piece of AV        material.

Correspondingly, a Core Metadata scheme according to the presentinvention, will have one class of Core Metadata named “Title”, and allthe above mentioned given metdata should be used as input to derive“Title” candidates, because—depending on what they contain—they mightcontain good material to be used for the above mentioned intuitivenotion of “Title”. Hence, the Core Metadata scheme will definetransformation rules for each of the metadata above, specifying in eachcase which part of the content should be inserted as the content of thederived Core Metadata item. The Core metadata scheme may then providetools such that the user can inspect all the “Title” candidates, selectone of them and/or edit their content as to match his/her personalpreference.

An even more illustrate example for the derivation of “Title” candidatescould be as follows. EPG data are transmitted according to theTV-Anytime standard and are received by a TV receiver or set top box. Auser invokes a display of the EPG on a television screen and selects atelevision program in the displayed EPG based on the displayed title.The title information according to the TV-Anytime standard is separatedand stored as a first candidate of the Core MD “title”. After tuning theTV receiver to the selected channel, the SI-information within the DVBsignal is analyzed and a second title information is separated andstored as a second candidate. Furthermore, the teletext signal receivedon the received channel is checked for information about the currentprogram, which is stored as third title candidate. Since the storedtitle information is based on different standards, differenttransformations are applied to the title informations in order toconvert them all into the same format. All converted title informationsare stored, and at any convenient later time, they can be displayed tothe user and the user then can select the “best” title information. Forexample, for a long title two of the three title informations may happento be abbreviated versions while only the third one may happen to becomplete. The user may in this case select the complete titleinformation for future overviews of his recordings.

Metadata being defined as “data about data”, a constituent part of anyoriginal metadata will always be an essence link, by which the metadatapoints to an AV essence. From this it is obvious, that parts of thetransformation rules MD-T will advantageously describe the details howthe essence links found on an input metadata shall be duplicated inorder to appear on the derived Core Metadata, too. In addition toessence links, an original metadata may also contain “origin links”specifying where the original metadata was received or extracted from.Assuming that Core Metadata, too, may have such an “origin link”,another part of the transformation rule MD-T may describe how this CoreMetadata origin link shall be set, one possible solution being that thislink points to the original metadata from which the core metadata wasgenerated.

For original metadata which are themself structured, like thoseaccording to the DVB-SI or TV-Anytime standard, the transformation rulesMD-T will typically select only certain kinds of metadata for furtherprocessing, will test for the presence of certain optional structuralparts of the input metadata, will potentially check some parts whetherthey have prescribed values, and, conditional on these preconditionsbeing met, the transformation will extract a certain subset of the inputmetadata and embed it in a certain position of the output Core Metadatato be generated.

For other input metadata, especially those which are unstructured exceptbeing indicated as “textual”, a transformation rule MD-T may typicallyconsist in evaluating a phrase similarity measure between the inputmetadata and some user-provided search phrase. Such a phrase similaritymeasure should advantageously be made tolerant against slight variationsof grammar or wording, which can be achieved by evaluating word orderand word distance, and by additionally converting all words to becompared into word stems using algorithms known in the art as“stemming”.

The invention is especially useful for personal video recording, e.g.for the search and access of specific recordings on high-volume storagemedia. However, the invention is also applicable to all other kinds ofelectronic multimedia content referencing and content location. Therecorded content may be movies and television programs but also everyother form of audiovisual information, such as arbitrary combinations ofstill pictures, graphics, 3D models, audio, speech and video.

1. Method for using metadata from different sources, said metadatadescribing and being linked to multimedia essence and being used foraccessing said multimedia essence, comprising receiving originalmetadata from different sources; deriving from said original metadataunified metadata using one or more transformation rules, saidtransformation rules specifying how the content or parts of the contentof said original metadata shall be used to constitute the content orpart of the content of said unified metadata.
 2. Method according toclaim 1, further comprising presenting said derived unified metadata toa user; processing said derived unified metadata in response to a userinput.
 3. Method according to claim 2, wherein said processing resultsin selecting, deleting or modifying said unified metadata.
 4. Methodaccording to claim 1 further comprising storing the original metadata;keeping the stored original metadata unchanged; and enabling to recurfrom the unified metadata to the original metadata.
 5. Method accordingto claim 1, wherein said original metadata contain links, and where apart of said transformation rules covers the mapping or inheritance ofsaid links from within said original metadata into all derived saidunified metadata.
 6. Method according to claim 1 wherein said originalmetadata are at least partly structured and where said transformationrules are at least partly based on examining and evaluating thestructure of said original metadata.
 7. Method according to claim 1,wherein said original metadata at least partly consist of sequences ofwords not otherwise structure, where said transformation at least partlyinvolves a search phrase, where said transformation rules are at leastpartly based on evaluating a phrase similarity measure, where saidphrase similarity measure is based on word order and/or word distancemeasures, and where, prior to applying said phrase similarity measure,the words of said original metadata and of said search phrase areoptionally normalized by applying a stemming algorithm.
 8. Apparatus forusing metadata from different sources, said metadata describingmultimedia essence and being used for accessing said multimedia essence,wherein means for receiving original metadata from different sources;means for deriving from said original metadata unified metadata usingone or more transformation rules said transformation rules specifyinghow the content or parts of the content of said original metadata shallbe used to constitute the content or part of the content of said unifiedmetadata.
 9. Apparatus according to claim 8, further comprising meansfor presenting said derived unified metadata to a user; means forprocessing said derived unified metadata in response to a user input.